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1.  Introduction 


Many  complex  stochastic  systems  of  interest  to  simulators  involve 
general  state  space  considerations.  For  example,  the  general  discrete- 
event  simulation,  when  viewed  as  a  generalized  semi-Markov  process,  is 
basically  a  discrete  time  Markov  chain  living  on  an  uncountable  state 
space. 

In  this  paper,  we  shall  consider  a  class  of  general  state  space 
Markov  chains  satisfying  a  certain  recurrence  property  known  as  Harris 
recurrence.  'We  shall  explore  these  chains  from  a  perspective  due  to 
Athreya  and  Ney  [2]  and  Numnelin  [17).  This  perspective,  while  natural 
in  its  own  right,  is  especially  appropriate  from  a  simulation  standpoint, 
as  it  essentially  tries  to  reduce  analysis  of  a  general  state  space 
chain  to  that  of  a  regenerative  process. 

We  start,  in  Section  2,  by  defining  the  abstract  notion  of  a  Markov 
chain  on  general  state  space.  Some  elementary  properties  of  such  chains 
are  then  explored,  and  several  examples  of  such  processes  are  introduced. 

In  Section  3,  we  give  Che  definition  of  Harris  recurrence.  We  then 
follow  the  method  of  Athreya,  Ney,  and  Nummelln  in  showing  how  a  certain 
subclass  of  such  processes,  called  strongly  recurrent  chains,  can  be 
embedded  in  a  regenerative  environment.  Consequences  of  the  embedding 
are  displayed,  and  the  examples  of  Section  2  are  analyzed  from  this 
recurrence  viewpoint.  We  also  give  a  means  of  reducing  many  weakly 
recurrent  (non  strongly  recurrent)  chains  to  a  strongly  recurrent 
process.  We  then  relate  our  concept  of  a  Harris  chain  to  the  classical 
one  (see  e.g.,  Harris  [13])  and  use  the  relationship  to  prove  results 


concerned  with  guaranteeing  Harris  recurrence  for  certain  types  of 
chains. 

Section  4  deals  with  a  historical  precursor  of  Harris  recurrence 
known  as  Doeblin  recurrence  (Doob  [ 9] ) .  The  relationship  between  Doeblin 
and  Harris  chains  is  investigated  from  the  Athreya-Ney-Nummelin  per¬ 
spective  introduced  in  Section  3.  A  result  showing  that  a  special  class 
of  Harris  chains  can  be  weakly  approximated  by  Doeblin  processes  is  then 
demonstrated. 

Ergodic  theory  for  Harris  chains  is  dealt  with  in  Section  5.  Emphasis 
is  placed  on  the  relationship  between  ergodic  behavior  and  certain  perio¬ 
dicity  and  uniformity  conditions.  Some  comments  are  also  made  on  strong 
laws  of  large  numbers  for  such  chains  that  pertain  to  simulation  of  such 
processes. 

Section  6  deals  with  regenerative  simulation  of  the  strongly 
recurrent  chains  defined  in  Section  3.  The  application  of  the  regenerative 
method  of  simulation  to  this  situation  is  outlined.  To  initiate  the 
regenerative  simulation,  a  certain  measure  is  required.  An  optimality 
condition  concerning  the  measure  is  considered ,  and  a  means  of  deter¬ 
mining  it  is  Indicated.  Also,  an  acceptance-rejection  method  intimately 
connected  with  the  basic  regenerative  framework  is  given.  Finally, 
practical  simulation  difficulties  involved  with  the  method  are  discussed. 

In  Section  7,  it  is  shown  that  a  broad  class  of  generalized  semi- 
Markov  processes  is  only  weakly  recurrent.  This  motivates  the  develop¬ 
ment  of  a  framework,  very  similar  to  that  of  Athreya,  Ney,  and  Nummelin, 
for  embedding  such  chains  in  a  "loosely  regenerative"  environment.  A 
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central  limit  theorem,  from  which  confidence  intervals  may  be  constructed, 
is  given  for  such  processes.  Finally,  the  necessary  modifications  in 
the  methods  of  Section  6,  for  simulation  of  weakly  recurrent  chains,  are 
outlined. 


2.  General  State  Space  Markov  Chains 

In  this  section,  we  Introduce  the  concept  of  a  Markov  chain  (M.C.) 
on  general  state  space.  He  then  proceed  to  explore  some  elementary 
properties  of  such  chains. 

Let  S  be  a  non-empty  set  and  S  a  o-field  of  subsets  of  S. 

A  function  P:S  *S  +  [0,1]  is  called  a  probability  transition  kernel 
ifj 

i)  P(x,-)  is  a  probability  measure  on  (S,S),  for  every  x  in  S. 
ii)  P(‘,B)  is  an  S-measurable  function,  for  every  B  (  S. 

The  n-step  transition  probabilities  are  defined  through  the 
relations 


(2.1) 


PX(x,B)  -  P(x,B> 

Pn+1(x,B)  -  /  P°(y,B)  P(x,dy)  , 
S 


n  >  1  . 


The  preceding  definitions  specify  the  state  space  and  finite-dimen¬ 
sional  characteristics  of  the  probability  transition  mechanism.  He  now 
turn  to  construction  of  the  process. 


i  1 

i  ' 
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t — r 


Let  2  ■  S  x  S  x  • • •  and  let  F  be  the  product  a-f ield 

S  x  S  x  ....  For  «  ■  (o)q,  ...)  €  2,  put  X^(«)  *  cu^.  Then,  for 

each  x  €  S,  there  exists  a  unique  probability  measure  on  (2,F) 

such  that  for  any  n  >  1  and  B. ,  B-,  ....  B  €  S, 

X  L  n 

P  {X.  €  B, ,  X„  €  B„,  ....  X  €  B  > 
xx  x  a  L  n  n 

-  /  P(x,  dy  )  /  P(y  ,  dy.)  •••  /  P(y  .,  dy  )  . 
a  x  R  x  i  _  n-i  n 

B1  B2  Bn 

Also,  as  a  consequence  of  the  measurability  of  the  kernel,  the 
map  x  *►  Px<E)  is  S-measurable,  for  every  E  €  F  (see  Revuz  [20], 
p.  16).  Hence,  we  can  define,  for  an  arbitrary  measure  n  on  (S,S), 
the  measure  P  on  (2,F)  via 

I* 

(2.2)  P  (E)  -  /  P  (E)u(dx) 

**  S 

Clearly,  for  ■  6^  (6x(B)  *  1  or  0  depending  on  whether  or 
not  x  €  B),  P  •  P  .  The  relation  (2.2)  extends  to  expectations  in 

r 

the  sense  that  if  Z  is  a  real-valued  F-measurable  function,  then 

(2.3)  E  Z  -  /  E  Z  |i (dx) 

^  S  X 

provided  that  at  least  one  side  of  the  equality  is  well-defined. 

It  can  be  readily  verified  that  the  process  {X  ;  n  >  0}  constructed 

n  — 

relative  to  (2,F,P^)  has  the  property  that 


4 


(2.  A) 


VVk  e  B|Fn>  ■  pk«„-B> 

a.s.,  where  F^  ■  8(Xq,  ...»  X^)  (the  smallest  o-field  generated 
by  Xq,  Xq).  This  is  just  the  statement  of  the  ordinary  Harkov 

property  (MP) . 

The  process  {XQ}  just  constructed  also  has  the  Strong  Markov 

property  (SMP).  To  be  precise,  let  9^:2  -►2  be  the  shift  given  by 

(2.5)  0  (to)  ■  (to i  »  to  .  - ,  ...  )  . 

n  n  nti 


We  say  that  T:2  -*■  {1,  2,  ...}  is  a  stopping  time  if  (T  *»  n}  €  F^.  The 
o-field  Ft  associated  with  T  is  given  by  the  set  of  events  E  €  F 
such  that  {T  -  n)  0  E  €  F  for  all  n.  Then,  the  SMP  states  that 


(2.6) 


yz-eTlFT>  » 


a.s.,  where  Z  is  any  real-valued  F-measurable  function  with 

E^lzl  < 

The  process  just  constructed  will  be  called  the  canonical  M.C. 
associated  with  P.  We  now  introduce  some  examples  of  such  processes. 


(2.7)  Example 

Let  S  *  {0,  1,  2,  ...)  and  suppose  P  »  (P^;  i,j  f  S)  is  a 
stochastic  matrix.  Then,  {X  }  is  a  countable  state  M.C. 


5 

_ i 


Consider  the  {Wn)  waiting  time  process  associated  with  a  G/G/l 
queue.  Let  U,V  be  random  variables  with  the  relevant  interarrival 
and  service  distributions,  respectively,  and  suppose  that  F(x)  =  P{V-U  <_  x) 
has  continuous  density  f(x).  The  state  space  here  is  S  =  [0,  +°°) ,  with 
S  the  Borel  sets  on  S.  Also,  for  B  €  S 

P(x,B)  =  HWn+1  6  B|Wn  -  x} 

-  Fife  +  vn  -  Vl)+  f  B> 

*  /  f(y)dy  +  6n(B)  I  f(y)dy 

B-x  (-*>,-x] 

where  B-x  =  {z:zfx  €  B}.  Henceforth,  by  {W^}  process,  we  shall  mean 
the  canonical  chain  associated  with  the  above  kernel. 

(2.9)  Example 

Let  Z^,  Z^,  ...  be  a  sequence  of  independent,  identically 
distributed  (i.i.d.)  random  variables  with  common  distribution  function 
F  having  support  on  [0,  2b],  b  >  0,  Also,  suppose  that  F  has  a 
continuous  density  f.  Define,  for  0  <  y  <  2b,  a  sequence  Xq,  X^,  ... 


through 


The  sequence  {X^}  can  be  realized  as  a  canonical  chain  relative  to 
(2,F,Py),  where  the  probability  triple  was  constructed  from  S  -  [0,  2bJ , 

S  «  Borel  sets  on  S,  and 

P(x,B)  =  P { | x-Z I  €  B> 

*  /  [f(x-y)  +  f(x+y)]dy  +  /  f(x+y)dy  . 

Bn[0,x]  Bfl[x,2b] 

Feller  [10],  p.  208,  can  be  consulted  for  further  details. 

(2.10)  Example 

A  class  of  stochastic  processes  that  appears  to  hold  promise  as 
a  mathematical  model  of  the  general  discrete  event  simulation  is  the 
generalized  semi-Markov  process  (GSMP).  A  GSMP  is  a  process  on  a  countable 
state  space  that  associates  with  each  state  a  set  of  clock  readings. 

When  the  clock  with  the  minimal  reading  runs  down  to  zero,  a  state 
transition  is  initiated.  The  process  jumps  to  another  state,  where 
the  clocks  from  the  preceding  state  continue  to  run  down,  and  where  new 
clocks  (including  possibly  the  clock  that  initiated  the  jump)  are  set 
stochastically  to  new  values.  This  procedure  then  repeats  itself  to 
generate  a  sample  path  of  the  GSMP. 

These  processes  can  be  analyzed  by  studying  an  associated  general 
state  space  Markov  chain,  namely  the  process  that  records  the  state  of 
the  GSMP  together  with  all  relevant  clock  readings,  at  the  time  of  each 
transition.  We  refer  the  reader  to  Whitt  [23]  for  a  detailed  description 
of  the  process,  and  for  the  notation  that  shall  appear  below. 
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The  state  space  S  of  the  M.C.  is  given  by  2,  the  set  of  all  GSMP 
states  together  with  all  possible  associated  clock  readings.  The  kernel 


l.  Harris  Recurrent  Markov  Chains 

We  note  that  for  Example  2.9,  there  exist  no  points  in  S  that 
are  visited  infinitely  often  (i.o.),  and  yet  one  can  reasonably  expect 
that  this  process  has  "nice"  ergodic  behavior.  Therefore,  any  good 
definition  of  recurrence  in  general  state  space  should  be  free  of  such 
an  assumption. 

We  say  that  a  M.C.  {X  }  is  Harris  recurrent  if  there  exists  a 

n 

tuple  (A,  B,  X,  cp,  n)  such  that 

i)  P  (T  <  +“>}  *  1,  for  every  x  €  S,  where  T  =  inf{n  ^  1:X  €  A} 
ii)  Pn(x,E)  ^  )up(E)  for  all  x  €  A,  measurable  E  c_  B,  where  X  >  0 
and  <p  is  a  probability  on  B. 

This  notion  of  recurrence  was  first  introduced  and  studied  by 
Athreya  and  Ney  [2]  and  Nummelin  [  17] . 


Remarks 


(3.1)  The  second  condition  is  a  smoothness  condition  on  the 
kernel.  The  first  condition  in  fact  guarantees  that  A  is  hit  i.o. 
Observe  that  the  consecutive  hitting  times  Tn  of  A  are  given  by 


T.  =  T.,  T  “  T  i  +  T  »0 
i  An  n-x  i  l 


Then 


n-1 


P  (T_  <  ■!“) 
X  2 


VT1  <  ■***•  v\ 

Ex(I(I,  <■•-}  PX  (T 
ll 

P  {T.  <  4«}  -  1 
x  1 


< 

1  <  4«}> 
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HaBSy* 


so  Chat  T2  ^  +00  a.s.  Now,  proceed  inductively. 

(3.2)  The  X  parameter  is  constrained  by  0  <  X  <_  1.  This 
follows  from 


1  2  pn(x  ,B)  2  *4>(B)  -  ^  >  0  . 


(3.3)  Note  that  condition  ii)  implies  that  tp(‘)  is  absolutely 
continuous  with  respect  to  Pn(x,-)  (<p  «  Pn(x,-))  for  every  x  €  A. 
Hence,  by  the  Radon-Nikodym  Theorem  (see  e.g.  Royden  [21]),  the  derivative 
of  c?  with  respect  to  (w.r.t.)  Pn(x,*)  exists  and  satisfies 


(3.4) 


(y)  <  1 


Pn(x,*)  a.e.  in  y. 


(3.5)  Without  loss  of  generality,  we  can  suppose  that  A  =  B 

in  the  above  definition.  In  other  words,  if  there  exists  (A,B,X,(p,n) 
satisfying  the  above,  then  there  also  exists  a  tuple  (A'  ,A'  ,X'  ,<p '  ,n' ) 
satisfying  i)  and  ii) .  In  fact,  one  can  take  A  =  A' .  See  Lemma  3.1 
for  a  proof. 

(3.6)  We  shall  call  a  set  A,  such  that  there  exists  X,  <p,  n 
with  (A,A,X,<p,n)  a  Harris  tuple,  an  A-set. 
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(3.7)  Lemma 

Suppose  (Xn)  is  a  process  with  kernel  P  satisfying  (A.B.X^n) 
recurrence.  Then  the  process  is  also  (A,A,X'  ,cp'  ,n' )  recurrent. 

Proof .  By  condition  i)  and  eqn.  (2.4) 

VTA  <  *“>  =  1  V»A  <  +->  <p(dx)  =  1 

s 

so  that  there  exists  an  integer  m  such  that 

Pm(x  (  A}  a  /  Pm(x,A)  (J>(dx)  >  0  . 

<P  m  $ 

Then,  for  x  in  A,  and  measurable  E  £  A,  we  can  utilize  (3.3)  to 
get 


Pn+m(x,E)  «  /  Pm(y,E)  Pn(x,dy) 

S 

>  X  /  Pm(y,E)  -  I  (y)  Pn(x,dy) 

s  Ldp  (*>•>  J 

»  X’  <p'  (E) 

where 

<p’(E)  -  /  Pm(y,E)  <p(dy)/P  {X  ?  A} 

S  9 

X’  «  XP  {X  €  A)  . 

tj>  m 
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This  verifies  the  second  condition  of  Harris  recurrence  for  the  tuple 
(A,A,V  ,<p'  ,n+m) .  The  first  condition  is  clearly  satisfied.  || 


We  return  to  our  examples  of  Section  2. 


Example  2.7  continued 


Suppose  {x  }  is  an  irreducible  recurrent  M.C.  Then  there 
n 

i  €  S  such  that  {i}  is  visited  i.o.  by  {X  }.  Here,  one  puts 

n 

B  **  S,  X  *  1,  <p  «  ,  n  ■  1  to  get  the  required  tuple. 


exists 
A  «  {i}. 


Example  2.8  continued 

We  know  that  if  EV  EU,  {0}  is  visited  i.o.  by  the  {W^} 

chain.  Here,  we  take  A  =  {0},  B  *  S,  X  *  1,  <p(E)  =  P(0,E),  n  *  1. 


Example  2.9  continued 

Suppose  that  f  is  positive  over  [0,2b],  so  that 
min{f(x):x  €  [0,2b]}  =  a  >  0.  Then,  putting  A  *  [0,b],  we  see  that  for 
any  Borel  subset  E  of  A,  and  x  in  A, 

P(x,E)  2  /  f (x+y)dy  >  am(E) 

E 

where  m  is  Lebesgue  measure.  It's  not  difficult  to  show  that  A  is 
visited  i.o.  and  so,  by  putting  B  *  A,  X  *  ab,  cp(*)  =  m(*)/b,  n  *  1, 
we  see  that  [X  }  is  Harris  recurrent. 


As  might  be  expected,  the  integer  n  of  the  condition  ii)  is 
closely  related  to  the  concept  of  periodicity  for  general  state  space 
chains . 

We  say  that  {Xq}  is  strongly  aperiodic  if  the  chain  is  (A,A,X,cp,l) 
recurrent.  To  define  further  notions  of  periodicity  for  Harris  chains, 
we  consider 

**A  8c<*{k  :  <Pk  with  {Xn}  (A.A.X^.cp^.k)  recurrent} 

for  A-sets  A.  It  can  be  shown  that  d^  is  in  fact  independent  of 
the  choice  of  A  (Athreya  and  Ney  {1],  p.  7).  It  is  then  said  that 
{Xn)  is  periodic  of  period  d^  if  dA  >  1,  and  aperiodic  if  d^  =  1. 
Given  that  {X^}  is  of  period  d,  the  state  space  S  can  be  decomposed 
into  sets  F,  C, ,  C,  such  that 

a1  * 

i)  F  =  S  -  U  C  and  P  {X  €  F  i.o.}  =  0  for  every  x  in  S. 
i-1  1  x  n 

ii)  C^,  ...,  Cd  is  a  cycle,  meaning  that 


p(x,  Cj+1)  *  1, 

x  €  Cj,  1  <  j  <  d-1 

P(x,  Cx)  *  1  , 

x  €  C , 
a 

(see  Orey  [19],  p.  13).  We  shall  see  in  Section  5  the  way  in  which 
periodicity  affects  the  ergodic  behavior  of  a  Harris  chain. 

We  will  now  indicate  how  the  Harris  condition  implies  nice  ergodic 
behavior  for  the  chain.  It  turns  out  that  when  a  kernel  P  is  (A,B,\,q»,l) 


recurrent,  then  one  can  embed  the  original  process  in  one  with  a  larger 
state  space,  such  that  the  new  process  is  regenerative.  One  then 
obtains  "nice"  ergodic  behavior  for  the  extended  process,  which  is 
subsequently  inherited  by  the  original  process. 

Under  the  condition  of  (A,B,X,ip,l)  recurrence,  we  can  decompose 

P  as 


(3.8) 


P(x,E)  -  Vp(E)  +  (1-X)  Q(x,E) 


for  x  €  A,  E  in  B.  The  function  Q  is  a  probability  transition 
kernel  on  (S ,5)  and  is  uniquely  determined  by  equation  (3.8)  for 
0  <  X  <  1  (for  X  =*  1,  Q  is  irrelevant). 

Athreya,  Ney,  and  Mummelin  exploited  the  decomposition  (3.8)  in 


P'((x,5),  E  x  {0»  * 


(3.9) 


P’((x,5),  E  x  {1}) 


=  S  X 

(0,1},  and 

S’ 

the 

associated 

pi 

relative  to 

(S', 

,S’) 

by 

(1-X) 

P(x,E)  , 

X 

t  A 

(1-X) 

Q(x,E)  , 

X 

€  A 

XP(x, 

E)  , 

X 

1  A 

Vp(E 

n  b)  , 

X 

€  A 

For  a  measure  p  on  (S ,S),  define  p'  on  (S’, S')  by 
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(3.10) 


j 

H'(E  x  U»  =  50(i)  (1-X)  n(E)  +  Sx(i)X  n(E)  .  ' 

t 

Then,  if  we  let  {X }  *  {(X  ,  5  )}  be  the  canonical  M.C.  corresponding 
u  n  n 

to  S'  and  P^, ,  it  can  be  shown  that:  ' 

f 

i)  for  any  p.  on  (S ,S) ,  the  marginal  distribution  (w.r.t.  P  ,)  of 

the  coordinate  process  {X  )  of  the  M.C.  (X*  >  and  the  distribution 

n  n 

(w.r.t.  P  )  of  the  original  M.C.  (X  }  are  identical. 

P  n 

ii)  the  random  variables  {6^}  constitute  a  sequence  of  i.i.d. 

Bernoulli  deviates. 

Remark 

(3.11)  The  correspondence  between  P  ,  and  P  means  that 

sample  path  properties  of  {X^j}  hold  under  P^,  if  and  only  if  they 

hold  under  P^.  For  example,  proving  a  strong  law  of  large  numbers  for 

(X  }  under  P' ,  means  that  it  will  also  hold  for  {X  }  under  P  . 
n  P  n  p 

Similarly, 

p'{f(X  ,  ....  X)  <  x}  -  F(x) 
l*u  n  — 

iff  P^{f(X0,  ....  Xn)  <  x}  *•  F(x) 

so  that  central  limit  theorems  will  also  coincide. 

Define  N  •*  inf{n  >  1:X  ,  (  A,  5  -  1}  and  put  N.  ■  inf {n  >  N.  .  :X  .  €  A, 

l  n-l  n  k  k-1  n-1 

6*1}  for  k  >  1.  Then,  for  every  k  >  1,  N.  <  -H»  p',  a.s.  and 
n  —  k  p  * 

j 

the  sequence  {N^;  k  >  1}  constitute  regeneration  points  for  the  j 
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« 

process  X^.  The  regenerative  property  stems  from  the  fact  that 


i 


€  E} 


<  E) 


I  t 


-  Plix'  €  E} 
*  n 


where  <j!(E  x  {i> )  *  6^1)  (p(E).  The  fact  that  X^  is  regenerative 

means  that  one  can  expect  strong  laws  of  large  numbers  and  central 

limit  theorems  to  be  valid,  under  reasonably  mild  conditions.  For 

example,  the  classical  regenerative  theory  (see  e.g.  Smith  [22]) 

shows  that  if  El,  N1  <  +°»,  then  the  strong  law 
<P 


(3.12) 


1 

n+1 


n 

l 

k-0 


:(v 


N,-l 


/  1 
-*■  Eif  l 
<P'  jt0 


f 


/eI  N 

<P  A 


holds  p',  a.s.,  provided  that  the  numerator  of  the  ratio  on  the  right 
hand  side  is  finite.  This  will  always  be  the  case  if  f  *  1^,  and  then 
(3.12)  becomes 


(3.13) 


n  Nrl 

sr  Jo  W  *  E;(  l  w)*;  -1 


Denote  the  limit  by  rr(A).  It  can  be  shown  (Athreya  and  Ney  [2],  p.  499) 
that  the  set  function  n(A)  is  a  probability  measure  satisfying 
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(3.14) 


n(E)  *  /  P  (x  ,E)  n(dx) 

S 

A  measure  satisfying  (3.14)  is  called  an  invariant  measure  for  P. 
Relation  (3.13)  is  then  a  classical  ergodic  theorem  stating  that  the 
time  average  of  the  process  tends  to  the  spatial  average. 

Remarks 

(3.15)  Using  a  slightly  more  sophisticated  argument,  it  can  be 
shown  (Athreya  and  Ney  [2],  p.  499)  that  every  Harris  chain  possesses  a 
o-finite  invariant  measure.  This  invariant  solution  is  unique  among 
all  o-finite  measures,  up  to  a  positive  multiplicative  constant. 

(3.16)  Relation  (3.13)  shows  that 

»rl 

*«>  -  Ei<vv>  i  Ei(  I  vy)  •  «<E>  Ej  »i 

and  hence  q>  «  ir.  This  will  also  be  true  of  general  Harris  chains. 

Let  us  say  that  a  chain  is  strongly  recurrent  if  it  is 
(A,B,\,<p,l)  recurrent.  The  following  example  shows  that  not  all  Harris 
chains  are  strongly  recurrent. 

(3.17)  Example 

Let  {X  }  be  an  (A,A,\,<p,l)  M.C.  on  (S,S)  with  kernel  P. 
n 

Then,  the  M.C.  X  *  (X  ,  X  ..)  has  state  space  S  x  S  and  kernel 
n  n  n+J. 
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P((x,y),  El  x  e2)  =  5y(E1)  P(y,  Ej)  . 

We  claim  that  #  is  Harris.  Observe  that  if  E^,  E2  £  then  for 
(x,y)  €  A  x  a, 

P2(Cx,y),  E1  x  E 2)  -  /  P (y,  dz)  P(z,  E2>  >  X2  tp^)  <p(E2)  . 

E1 

Furthermore,  it  is  clear  that  A  x  A  is  visited  i.o.  by  the  process 

A  A  2 

X,  and  so  X  is  (A  x  a,  A  x  a,  X  ,  ip  x  <p ,  2)  recurrent. 

A 

Now,  suppose  that  P  is  a  strongly  recurrent  kernel,  say 
(B^,  B2>  a,  \^,  1)  recurrent.  Then,  since  t  <<  P((x,y),  •)  (Remark  (3.3)) 
for  some  (x,y),  must  be  supported  by  {y >  x  s.  Under  the  assumption 
of  strong  recurrence  the  chain  must  be  able  to  pass  from  B^  into 
{y}  x  s  in  one  step.  Consequently,  B^  £  S  x  {y}.  However,  it  is 
clear  that  in  general  X  will  not  hit  a  B^  of  this  form  i.o.  Hence, 

A 

X  will  not,  in  general,  be  strongly  recurrent. 

Although  Harris  chains  are  not  always  strongly  recurrent,  one 
can  often  modify  the  process,  in  a  natural  way,  so  that  the  resulting 
chain  is  strongly  recurrent. 

A 

Given  a  kernel  P,  we  say  that  P  is  the  randomized  version  of 
P  if  it  has  the  form  (0  <  p  <  1) 

P(x,E)  -  l  p(l-p)k  1  Pk(x,E)  . 
k-1 
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(3.18)  Lemma 

Let  P  be  a  Harris  kernel  with  invariant  probability  n.  Then, 

P  is  a  strongly  aperiodic  Harris  kernel. 

Proof :  Suppose  that  P  is  (A,  A,  X,  fp,  n)  recurrent.  Then, 
for  x  €  A  and  E  £  A, 

P(x,E)  >  p(l-p)n-1  pn(x,E) 

>  P(l-P)11"1  Xp(E) 

so  that  P  satisfies  the  second  condition  of  (A,  A,  p(l-p)  X,  <p,  1) 
recurrence.  We  also  need  to  show  that 

P  {Ta  <  4-}  =  1 
x  A 

for  every  x  €  S.  With  the  existence  of  tt,  this  fact  is  proved  in 
the  Appendix.  || 

Remark 

(3.19)  The  P  chain  is  obtained  from  the  P  chain  by  sampling 
from  the  original  P  chain  according  to  a  sequence  of  i.i.d.  Bernoulli 
random  variables  of  parameter  p.  This  motivates  use  of  the  term 
'randomized'  version. 


■g; 


We  shall  now  describe  a  notion  of  recurrence  that  is  virtually 
equivalent  to  that  described  above.  This  definition  requires,  however, 
a  separability  assumption  on  the  a-field  S.  Therefore,  we  will 
assume  throughout  the  remainder  of  this  paper  that  S  is  countably 
generated. 

Remarks 

(3.20)  The  separability  assumption  is  automatically  satisfied  if 
S  is  the  Borel  o-field  associated  with  a  separable  metric  space  S. 

This  is  because  the  o-field  is  generated  by  the  open  balls  with  rational 
radii  centered  at  points  lying  in  some  countable  dense  subset  of  S. 

(3.21)  The  separability  assumption  can  be  avoided  to  some  extent 
by  using  the  concept  of  admissible  o-fields  (see  Orey  [19],  Doob  [9]). 

A  chain  [X  }  is  said  to  be  Harris  v-recurrent  if  there  exists 
n 

a  o-finite  measure  v  on  (S,S)  such  that  v(E)  >  0  implies 

Px{TE  <  ■*">  =  1 

for  every  x  in  S  (see  e.g.,  Harris  [13]). 

We  claim  that  under  the  assumption  of  separability  of  S,  Harris 
v-recurrence  and  Harris  (A,B,X,(p,n)  recurrence  apply  to  exactly  the 
same  class  of  chains.  First,  suppose  that  the  chain  {X^}  is 
(A,B,X,<p,n)  recurrent.  Then,  the  process  is  clearly  Harris  cp-recurrent. 
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For  the  converse,  we  use  the  concept  of  C-sets  (Orey  [19],  p.  7). 

By  way  of  Introduction,  we  recall  that  for  every  positive  integer  n, 
we  can  decompose  Pn(x,*)  into  its  v-singular  and  v-absolutely 
continuous  parts,  to  yield 

(3.22)  Pn (x,E)  =  /  pn(x,y)  v(dy)  +  p"(x,E) 

E  S 

where  P^(x,>)  is  the  v-singular  part  of  the  decomposition.  A  set 
C  £  S  is  said  to  be  a  C-set  if  v(C)  >  0  and  if  there  exists  m 
such  that 

(3.23)  inf  pn(x,y)  =  X  >  0  . 

(x,y)  €  Cx  c 

Let  0  <  v (E) <  +“.  Then  the  separability  of  S  allows  one  to  find  a 

well  enough  behaved  version  of  pn(x,y)  that  a  C-set  C  contained 

in  E  can  be  produced.  Observe  that  v(C)  >0  so  that  {X^}  visits 

C  i.o.  Furthermore,  for  x  in  C  and  B  in  C, 

Pn(x,B)  2  Pn(x,y)  v (dy)  2  ^v(B) 

E 

so  that  {X  >  is  (C,  C,  \v(C),  v(*)/v(C),  n)  recurrent, 
n 

Remarks 

(3.24)  The  concept  of  Harris  v-recurrence  is  in  fact  the  classical 
definition  of  Harris  recurrence  as  given,  for  example,  in  Orey  [19]  and 
Revuz  [20]. 
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(3.25)  Note  that  {X^}  is  always  <p-recurrent  under  the  cp  of 
an  (A,  B,  X,  (p,  n)  tuple.  However,  given  that  {X^}  is  v-recurrent, 
there  does  not  generally  exist  A,  B,  X,  n  such  that  {Xn>  is 
(A,  B,  X,  v,  n)  recurrent. 

We  now  give  some  indications  of  the  generality  of  Harris  recurrence 
First,  we  define  the  notation,  for  p  a  measure  on  (S,S), 

pf  =  /  f (y)  p(dy)  . 

S 


(3.26)  Lemma 

Let  {X^}  be  a  M.C.  with  the  following  ergodic  behavior.  Suppose 
that  there  exists  a  probability  tr  such  that  if  f  is  a  bounded 
S-measurable  function,  then 

1  n 

(3.27)  — rr  I  f(X.)  nf  ,  P  a.s. 

k=0  K  x 

for  every  x  in  S.  Then  {X^}  is  Harris  recurrent. 

Proof.  For  E  €  S,  take  f  =  I£.  Then, 

1ST  j„  W  *  “«>  •  px  *'s' 

k=0 

00 

so  that  if  n(E)  >  0,  then  Px{  £  I^X^)  =  =  1  for  every  x.  But 

k*»0 


22 


{  iE(\)  =  w  =  (y  e  i.o.> 


so  {X  }  is  Harris  rr-recurrent. 
n 


Remark 


(3.28)  Above,  we  showed  that  (3.27)  is  sufficient  for  Harris 


rucurrence.  In  Section  5,  we  will  shows  it  is  also  necessary,  in 


some  sense. 


We  will  now  apply  Lemma  3.26  to  obtain  Harris  recurrence  for  a 
fairly  general  class  of  chains.  Let  S  be  a  separable  metric  space, 
with  5  its  Borel  sets.  Put 


bS  =  class  of  bounded  S-measurable  functions 
bC  =  class  of  bounded  functions  continuous  with  respect 
to  metric  topology  on  S. 


One  can  regard  P  as  an  operator  on  bS  through  the  relation 


(Pf)(x)  =  /  P(x,dy)  f  (y)  . 
S 


We  will  say  that  P  is  a  Feller  kernel  if  P  maps  functions  from  bC 
back  into  bC. 
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— ? 


(3.29)  Lemma 

P  is  Feller  if  and  only  if  x  -*•  x  implies  P(x  ,  •)  *»  P(x,*) 

n  n 

(=»  is  weak  convergence  of  probabilities). 

Proof .  Now,  PCx^,-)  =>  P(x,  •)  if  and  only  if  for  every  f  €  bC 

/  f  (y)  P(x  ,dy)  -v  /  f  (y)  P(x,dy)  i.e.,  (Pf)(x  )  -  (Pf)(x)  . 

S  n  S  n 

But  this  is  precisely  the  notion  of  continuity  for  Pf.  The  boundedness 
of  Pf  is  trivial. 

Remark. 

(3.30)  It  is  easily  checked  that  the  kernels  P  of  Examples  2.7  -  2.10 
are  Feller. 

(3.31)  Lemma 

Let  S  be  a  compact  metric  space,  and  suppose  P  is  Feller  on 
(S,S).  Also,  suppose  that  P  possesses  a  unique  invariant  probability  tt 
that  assigns  zero  mass  to  SA,  for  any  A  €  S  (9A  is  the  boundary  of 
A).  Then,  P  is  a  Harris  kernel. 

Remark 

(3.32)  Every  compact  metric  space  is  separable  (see  e.g. ,  Copson 
[6],  p.  77). 
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Proof .  We  first  define  the  random  measures 


n 


Hn(E,<o) 


-JL  i 

11+1  k=0 


.(\<«)> 


We  next  employ  a  theorem  of  Breiman  [4],  which  states  that  under  our 
hypotheses 

T7T  l  fCMco))  =  f  f(y)  M  (dy,  w)  -►  nf 
k=0  *  S 

a.s.,  for  every  x  in  S  and  f  €  bC.  Now  bC  is  a  separable 
metric  space  (Remark  3.33)  under  the  sup  norm  metric  so  that  there 
exists  a  countable  family  C  of  functions  dense  in  bC.  Then  Px(D  * 
where 


T  =  {to:/  f(y)  M  (dy,  w)  nf,  V  f  €  C) 

S  n 

For  oo  €  T,  M  (•  ,co)  =»rr(’)  (Remark  3.34)  and  so,  because  it  (BA)  *  0 
n 

for  all  Borel  sets  A,  we  get  (Billingsley  [3] ,  p.  12) 


M  (E,oo)  -*•  it  (E) 
n 


for  every  E  in  S.  Consequently, 


1 

n+1 


k-0 


W 


n  (E) 


a.s. 


for  every  x  in  S,  which  implies  that  {Xfl}  is  Harris  n-recurrent 
(see  Lemma  3.26). 
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Remarks 


(3.33)  Let  {sn}  be  dense  in  S  and  let  {r^}  be  an  enumeration 
of  the  positive  rationals.  Then,  putting 

g_  _00  =  [r  -  P(x,s  )]  + 
n»n  m  n 

the  family  F  *  {g  }  is  a  countable  family  of  functions  separating 
01)11 

points  in  S.  Then,  every  function  in  bC  can  be  uniformly  approximated 
by  a  polynomial,  with  rational  coefficients,  in  a  finite  number  of 
functions  of  F  (Royden  [21],  p.  175).  This  latter  family  is  countable. 

(3.34)  Let  Q  ,  Q  be  probabilities  and  suppose  Q  f  -*■  Qf  for 

n  n 

all  f  in  some  subset  C  which  is  dense  under  the  sup  norm  ||*ll  in 

bC.  Then  Q  *»  Q  since  for  f  €  bC,  there  exists  f  in  C  such 
n  n 

that  ||f  — f ||  0  and 

n 

IV  - «fl  i  'V  -  VJ  +  i Vn  -  +  io£„  -  ofl 

<  2||£„-£|l  +  I  Vn  -  QV 

which  tends  to  zero  by  first  letting  m  -*•  «,  and  then  letting  n  +°°. 

(3.35)  Note  that  every  Feller  kernel  on  compact  metric  space 
possesses  an  invariant  probability  (Karr  [14],  p.  45)  so  that  the 
hypotheses  of  Lemma  3.31  really  entail  only  unicity. 
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(3.36)  An  easy  sufficient  condition  for  n (3A)  =0  is  that  there 
exists  n  such  that  Pn(x,*)  <<  p.  for  every  x,  where  p(3A)  «  0. 

Example  2.9  continued 

As  an  easy  application  of  Lemma  3.31,  note  that  P  is  clearly 
Feller.  In  addition,  P(x,*)  <<  ra  for  every  x  in  S,  and  so 
ir(3A)  =  0  (Remark  3.36).  Unicity  of  n  follows  as  a  consequence  of 
m(E)  >0  implying  P2(x,E)  >  0. 


4.  Doeblin  Recurrent  Markov  Chains 

Historically,  the  study  of  Doeblin  chains  preceded  that  of  Harris 
processes.  We  say  that  a  chain  {X^}  is  Doeblin  recurrent  if  there 
exists  a  tuple  (t,  y,  t),  n)  consisting  of  a  probability  T,  reals 
Y  <  1  and  T]  >  0,  and  an  integer  n  such  that,  for  E  €  S, 

(4.1)  t(E)  >  y  implies  Pn(x,E)  >  tj 

for  all  x  €  S.  Fossett  ([11],  p.  11)  has  shown  that  this  condition 
is  equivalent  to  that  of  Doob.  Doob's  condition  states  that  {X^}  is 
a  chain  with  a  tuple  (\|r,  e,  k)  consisting  of  a  finite  measure  +» 
a  positive  e,  and  an  integer  k  such  that  for  E 


(4.2) 


*(E)  <  c 


implies  P^(x,E)  <  1-e 


for  all  x  (  S.  The  measures  ^  and  x  in  Che  two  definitions  are 
equivalent  in  the  sense  that  each  can  play  the  role  of  the  other 
(provided  is  normalized  to  a  probability). 

The  concept  of  Doeblin  recurrence  is  closely  related  to  that  of 
Harris  recurrence.  Neveu  ([16],  p.  185)  noted  that  if  a  process  is 
(S,  S,  X,  <p,  n)  recurrent,  then  it  is  Doeblin  recurrent.  For  if 
we  choose  e  so  that  0  <  e  <  1,  then  <p(E)  >  c  implies 

(4.3)  Pn(x,E)  >  hp(E)  >  Xe 

for  every  x  in  S,  so  that  (cp,  e,  Xe,  n)  is  a  legitimate  Doeblin 
tuple. 

However,  not  every  Doeblin  process  is  Harris  recurrent. 

(4.4)  Example 

Take  S  *  {1,  ...,  n}  and  let  P  be  a  stochastic  matrix  on  S. 
Then,  if  we  put  x^  *  1/n  for  all  i,  we  see  that  x(E)  >  (n-l)/n 
implies  that  E  ■  S,  and  so  P(i,E)  =  1.  So  every  finite-dimensional 
stochastic  matrix  is  Doeblin. 

On  the  other  hand,  let  v  be  any  cr-finite  measure  on  S.  Then 

there  exists  1  such  that  v.  >  0,  and  so,  for  {X  }  to  be  a 

J  n 

v-recurrent  chain,  we  must  have 

P1(Tj  <■*-}-! 
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for  all  i €  S.  So  {X  }  must  be  an  irreducible  chain  in  order  that 

n 

P  be  Harris  recurrent  (this  is  also  sufficient,  by  Example  2.7, 
see  Section  3) . 

Example  4.4  points  to  the  difficulty,  namely  that  we  must  restrict 
attention  to  irreducible  chains  in  order  to  establish  a  proper  connection 
between  Harris  and  Doeblln  chains.  We  first  discuss  the  appropriate 
generalization  of  irreduclbillty  to  a  general  state  space  S. 

A  set  F  €  S  is  called  a  transient  set  if 

lim  Pn(x,F)  -  0 
n  -*■  00 

for  every  x  in  S.  Doob  showed  ([9] ,  p.  210)  that  for  a  Doeblin  chain, 
there  exists  a  decomposition  of  S  into  disjoint  ergodic  sets 
E^,  Ej,  ...  and  a  transient  set  F  *  S  ~  U E^.  An  ergodic  set  is  a 
set  E  for  which  there  exists  a  probability  n£  on  E  with 

.  n  . 

(4.5)  lim  i  l  PK(x ,B)  -  n_(B) 

n  h.  =o  n  k-1  E 

for  every  x  in  E  and  B  in  E.  The  ergodic  sets  determined  by  the 
decomposition  are  unique,  provided  points  in  transient  sets  are  neglected. 

The  way  to  generalize  irreducibility  to  this  setting  is  to  suppose 
that  there  exists  a  single  ergodic  set.  This  is  in  fact  equivalent  to 
assuming  that  {Xn>  has  a  unique  invariant  measure  (Doob  [9],  p.  214). 
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(4.6)  Lemma 


Let  (Xn>  be  a  Doeblin  chain  with  single  ergodic  set.  Then,  {X^} 
is  Harris  recurrent. 

Proof .  Let  n  be  the  unique  invariant  probability  of  {X^}. 

Then  we  can  use  Theorem  6.2  of  Doob  [9]  (p.  220)  to  conclude  that  if 
n  |  f  |  <  +«»,  then 


f(\)  ♦  rtf 


P a.s.,  for  every  x  in  S.  Application  of  Lemma  3.26  then  yields 
the  result.  || 


Given  that  a  Doeblin  chain  is  Harris,  how  does  the  process  fit  into 
the  Harris  framework?  The  following  two  lemmas  address  this  question. 


(4.7)  Lemma 

A  chain  {X^}  is  Doeblin  recurrent  with  a  single  ergodic  set  if 
and  only  if  the  chain  is  (A,  A,  X,  <p,  n)  recurrent,  and  there  exists 
an  integer  m  such  that 

sup  P  (T  >  m}  <  1 
x  €  S  X  A 


(4.8)  Lemma 

A  process  Xn  is  an  aperiodic  Doeblin  chain  with  single  ergodic 
set  if  and  only  if  the  chain  {X^}  is  (S,  S,  X,  tp,  n)  recurrent. 
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We  defer  the  proof s  of  these  results  to  Section  5. 

Remark 

(4.9)  Chains  of  the  type  described  in  Lemma  4.7  correspond 
precisely  to  the  uniformly  v-recurrent  chains  of  Orey  [19].  This 
fact  can  be  proved  by  employing  C-sets, 

Suppose  that  a  chain  is  (S,S,X,<p,n)  recurrent.  Then  the  tuple 
(cp,&,\e,n)  is  a  legitimate  Doeblin  tuple  (see  (4.3))  so  that  <p  can 
be  used  in  the  role  of  t  (see  (4.1)).  The  converse,  however,  is  false, 
as  the  following  example  shows. 

(4.10)  Example 

Let  S  =  {0,  1,  2,  ...}  and  let  P  be  given  by 
P(i,E)  «  |  5Q(E)  +  |  5.+1(E)  . 

This  chain  is  clearly  (S,S, 1/2, 5^,1)  recurent,  and  it  is  easily 
checked  that  rr^  *  l/2*+*  is  an  invarant  probability  for  P.  Furthermore, 
n (E)  >  1/2  implies  that  0  €  E,  so  that 

P(i,E)  >  1/2 

for  all  i.  Hence,  (tt,  1/2,  1/2,  1)  is  a  legitimate  Doeblin  tuple. 

On  the  other  hand,  there  can  exist  no  X  and  n  such  that  P 
is  (S,  S,  X,  tt  ,  n)  recurrent.  For  if  such  X  and  n  existed,  then 
we  would  have 
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Pn(i,j)  >  Xnj  >  0 

for  every  i  and  j .  But  this  contradicts  the  fact  that 
Pn(i,  i+n+1)  -  0. 

This  shows  that  although  n  is  a  legitimate  t  measure,  it 
is  not  a  legitimate  <p  measure. 

We  close  this  section  with  a  weak  convergence  theorem  for  Doeblin 
chains.  To  be  precise,  let  S  be  a  complete  separable  metric  space 
with  S  its  Borel  o-field.  Then,  by  defining  the  product  topology 
on  S2=SxSx**-,2  itself  becomes  a  complete  separable  metric 
space.  In  addition,  it  can  be  shown  (Dellacherie  and  Meyer  [8],  p.  11) 
that  the  a-field  F  is  precisely  the  Borel  a-field  under  this  topology. 

We  shall  say  that  a  kernel  P  is  strong  Feller  if  P(*,E)  is  a 
continuous  function  for  each  E  €  S.  Our  result  is  that  if  P^  is  a 
probability  on  (S2 , F)  constructed  from  a  Harris  recurrent  strong 
Feller  kernel  P,  then  one  can  find  a  sequence  of  probabilisties  Pr 
such  that 

i)  P  is  of  the  form  P^n\  where  n  a  ii  and  are  Doeblin 

n  u  n 

n 

kernels . 

ii)  P^  =»  P^  on  (S2,F),  i.e.,  for  every  bounded  function  h  continuous 
w.r.t.  the  topology  on  $2, 

/  h(co)  P  (dco)  ■+■  /  h(co)  P(dco) 
r.  n  o 
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We  start  by  recalling  that  since  P  is  v-recurrent,  v(A)  >  0 

implies  that  A  is  hit  i.o.  P  a.s.  For  such  an  A,  let  us  denote 

x 

the  consecutive  hitting  times  by  T^,  Tj,  ....  Then,  the  sequence 
(Kj,  }  is  a  M.  C.  with  kernel 


(4.11)  P  (x.E)  »  P(x,  E  n  A)  +  /  P (x,  dy)  P(y,  E  fl  A) 


+  /  /  P(x,  dy)  P(y,  dz)  P(z,  E  D  A)  +  ••• 

A  A 


This  chain  is  called  the  "process  on  A".  Orey  ([18],  [19])  showed 

that  for  a  v-recurrent  P,  there  exists  a  sequence  A  of  sets 

n 

increasing  to  S  (i.e.,  A  c  A  _ ,  UA  -  S)  such  that  P.  is  Doeblin 

n  —  n+i  n  A 

n 

on  An  (in  fact,  uniformly  v-recurrent).  We  construct  Pn  by  noting 

first  that  since  A  *  S,  p.(A  )  1  so  that  we  may  suppose  p.(A  )  >  0 

n  n  n 

for  each  n.  Then  put 


P 


(n) 


(x.E)  =  PA  (x , E) 


u  (E)  -  p(E  n  A  )/n(A  ) 
n  n  n 

and  let  P  ■  P^n^  corresPonding  measure  on  (S2,F).  We  now 

employ  a  weak  convergence  theorem  of  Karr  [14]  that  states  that  to 

show  P  »  P  ,  it  is  sufficient  to  prove  that 
n  p 

O 

ii)  x  x  in  S  implies  P^(x  ,•)  =»  P(x,*). 
n  n 
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Clearly,  i)  is  trivial.  As  for  ii) ,  note  that  for  f  €  bC, 

|P(n)  f(x  )  -  Pf  (x)  | 
n 

<  |p(n>  f(x)  -  /  f(y)  P(x  dy)| 

I  n  .  n  I 

A 

n 

+  /  f(y)  P(x  ,  dy)  -  Pf  (x  )  +  |Pf  (x  )  -  Pf  (x)  |  . 

I .  n  n  n 

An 

It  is  easily  checked  that  the  first  term  on  the  right  is  bounded  by 

||f||  P  {T  >  1}  *  ||f||  P(x  ,  AC)  (see  (4.11)).  The  second  term  is 
x  A  n  n 

n  n 

bounded  by  the  same  quantity.  So,  fixing  m  for  the  moement,  we  see 
that  for  n  _>  m 

|p(n)  f  (x  )  -  Pf  (x)  I  <  2||f  II  P(x  ,  A^)  +  |Pf  (x  )  -  Pf  (x)| 
n  —  n  m  n 

-  2 1| f  ||  P(x,  A^) 
m 

as  n  ■*■  *,  since  P  is  strong  Feller.  Then,  let  m  ■+■  +°°  to  complete 
the  proof  of  ii). 

Remark 

(4.12)  Karr  [14]  also  gives  conditions  under  which  the  invariant 
measures  of  a  sequence  of  M.C.'s  tends  to  an  invariant  measure  for  the 


limit. 


.  Ergodlc  Behavior  of  Harris  Chains 


The  ergodic  behavior  of  Harris  chains  is  basically  determined  by 

periodicity  considerations  and  by  certain  types  of  uniformity  conditions 

on  the  (A,  B,  \,  <p,  n)  tuple.  We  recall  that  every  Harris  chain  has 

a  unique  invariant  measure  (Remark  3.15).  Assuming  now  that  this  measure 

is  a  probability  n ,  our  first  goal  is  to  examine  convergence  to  tt  of 
1  2 

the  iterates  P  ,  P  ,  ...  of  the  transition  kernel. 

Then,  if  {X^}  an  aperiodic  Harris  chain 

(5.1)  (Pnf)(x)-TTf 

for  every  x  in  S  and  f  in  bS.  In  fact,  a  stronger  result  holds. 
Let  ||*||  be  the  L°°  norm  on  bS.  Then,  there  exist  constants  c^  +  0 
(independent  of  f)  such  that 

(5.2)  |(Pnf)(x)  -  f|  <  cjlfll  . 

If  the  process  is  periodic,  then  one  gets  convergence  of  the  averages, 
i.e.,  there  exist  constants  c^  4-  0  such  that 

I  .  n  .  I 

(5.3)  \~  l  (PKf)(x)  -  rrf  <  cjlfll  . 

Ini  k=0  1 

When  no  invariant  probability  exists,  one  can  still  prove  that 
if  P  is  an  aperiodic  Harris  kernel,  then  there  exist  constants  cn  4-  0 
such  that 
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V 


np 


W 


m 


pnf  -  ti2  Pnf  I  £  cj|f  II 

where  p.^,  are  any  two  initial  probabilities  on  (S,S).  Here  the 
operator  pPn  is  given  by 

pPnf  =  /  p(dx)  (Pnf)(x)  . 

S 

Proofs  for  these  results  can  be  found  in  Athreya  and  Key  [1]  or  Orey 
[19]. 

We  can  get  even  stronger  results  than  (5.2)  by  imposing  uniformity 
conditions.  Athreya  and  Ney  ([2],  p.  496)  consider  a  strongly  aperiodic 
(A,  A,  X,  <p,  1)  process  with  the  additional  restriction  that  there 
exist  an  integer  m  such  that 

sup  P  {T.  >  m)  <  1 
x  €  S  x  A 

They  show  that  for  such  a  chain,  there  exists  an  invariant  probability 
it  and  constants  b  and  p  (with  0  £  p  <  1)  such  that 

(5.4)  ||pnf  -  Ttf I!  £  bpn||f||  . 


Remark 

(5.5)  We  recall  from  (4.5)  that  all  Doeblin  chains  with  single 
ergodic  set  have  a  unique  invariant  probability.  If  such  a  chain  is 
also  aperiodic,  then  Doob  [9]  has  shown  that  there  exist  constants 
and  p  (with  0  £  p  <  1)  such  that 
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(5.6) 


lPn(x,E)  -  TT (E)  |  _<  ypn 
for  every  x  in  S,  E  €  $. 

The  similarity  between  (5.4)  and  (5.5)  provides  the  key  to  the 

A 

proofs  of  Lemmas  4.7  and  4.8.  The  randomized  kernel  P  will  also  be 
used  here.  Note  that  if  it  is  invariant  for  P,  then  it  is  invariant 
for  P. 

Mi 

On  the  other  hand,  suppose  that  rr  is  invariant  for  P.  First 
of  all,  a  glance  at  the  proof  in  the  Appendix  shows  that  the  assumption 
of  an  invariant  probability  for  P  made  in  Lemma  3.18  was  asserted  only 

A 

to  guarantee  existence  of  an  invariant  probability  for  P.  Hence, 

A  A 

given  that  P  is  Harris,  P  is  also,  and  so  P  has  a  unique  invariant 

A  A 

probability,  namely  rr.  But  rrP  =  rr  implies  that  (nP)P  =  rrP,  and  so 

A 

unicity  of  invariant  probability  for  P  implies  that  rrP  =  it.  Hence, 

A 

provided  P  is  Harris,  invariant  probabilities  for  P  and  P  coincide. 

Proof  of  Lemma  4.7 

=>)  Since  {X^}  is  Doeblin  with  single  ergodic  set,  P  is  Harris 

A 

with  invariant  probability  ti,  and  hence  P  is  Harris  (Lemma  3.18). 

Now,  supposing  that  P  offers  a  (t,  y,  r),  m)  Doeblin  tuple,  we  have 
t(E)  >  y  implies  Pm(x,E)  >  r)  so  that 

P(x,E)  >  p(l-p)m~1  Pm(x,E)  >  p(l-p)m“1  Tj 
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A 

and  hence  P  is  Doeblin  with  (t,  y»  p(l-p)  i),  D  tuple.  Now,  by 

A  A 

Lemma  3.18,  P  is  an  aperiodic  Harris  chain.  Also,  because  P  is 
Doeblin  with  unique  invariant  probability,  it  must  be  that  it  has  a 
single  ergodic  set.  Summarizing,  P  is  an  aperiodic  Doeblin  kernel 
with  single  ergodic  set. 

So,  we  can  apply  Remark  5.5  to  conclude  existence  of  a  y  and  p 
such  that 


|Pn(x,E)  -  ti(E)  |  <  yp"  • 


Now,  P  is  Harris  so  there  exists  an  (A,  A,  X,  <p,  k)  tuple  with 
rr  (A)  >  0.  Then,  we  can  find  n  such  that 


Pn(x,A)  >  n(A)/2 


i.e. , 


(5.7)  l  a.  Pj(x,A)  2  ff(A)/2 

j=n  3 


where 


<V 


is  a  probability. 


Choose  l  so  that 


l  a  <  rr  (A)/4  . 

j=£+l  3 


Then,  (5.7)  becomes 


][  a .  P^  (x,A)  2  n  (A) /A 
j=n  3 
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<*■  ,m  f 


Letting  a  =  maxia^;  n  j  £},  we  get  the  inequality 

t 

(5.8)  I  P^x.A)  >  rr(A)/4a  . 

j=n 

Putting  Sj  =  (x:P^(x,A)  _>  it  (A> / 43-8 > ,  we  see  that 
clearly  implies  that 

P  {T.  >  1}  <  1  -  n(A)/4a«  . 

x  A  — 

<=  The  uniformity  condition  on  P  requires  that 


U  S .  =  S ,  which 

j=l  3 


inf  Px{TA  <  m)  =  6  >  0 


and  hence 


l  Pk(x,A)  >  6 
k=l 


for  every  x  in  S.  If  we  now  form  the  sets  S  =  {x  :  PJ(x,A)  >  5/m}, 
m  ^ 

we  observe  that  U  S.  =*  S. 

j=l  2 

Then,  for  x  €  S.,  we  obtain  the  lower  bound 
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Pnri'n(x,E)  >  /  pm+n-j  (y >E)  Pj(x,dy) 

/  Pn(y,dz)  Pm_j(z,E) 

A 

/  Xq>(dz)  Pm_j(z,E) 

A 

Pm_^ (z,E)  , 

the  assumption  of  (A,  A,  X,  <p,  n)  recurrence  having  been  invoked  in  the 
third  inequality  in  the  chain. 

We  now  define  the  probability 

t(E)  *  —  l  /( p(dz)  Pm~j(z,E)  . 

“  k=l  A 

The  inequality  t(E)  _>,  1“Y>  where  0  <  y  <  min (1/m,  1/2),  then  yields 

^  /  <p(dz)  Pm  ^  (z,E)  >;-y 
m  ,  —  m 

A  L 

for  j  =  1,  m.  Consequently,  it  must  be  that 

Pm+n(z,E)  >  —  (1-Ym)  =  5*  . 

—  m  i 

We  may  therefore  conclude  that  (x,  1-X,  5*,  m+n)  is  a  legitimate  Doeblin 
tuple. 


A 

>  /  Pj(x,dy) 
A 

>  /  Pj (x,dy) 
A 

>  “  /  cp(dz) 
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Proof  of  Lemma  4.8 


=0  By  Lemma  4.6,  {X  }  is  Harris  recurrent  so  there  exists  an 

n 

(A,  A,  X,  (p,  n)  tuple  with  n (A)  >  0.  The  aperiodicity  of  the  chain 
allows  one  to  use  (5.5)  to  conclude  that  there  exists  an  integer  m 
such  that 


Pm(x,A)  >  tr(A)/2  =  5  . 

Then,  for  every  x  in  S  and  E  in  A,  we  get 

P^x.E)  =  /  Pn(y ,£)  Pm(x,dy) 

S 

>  I  Pn(y,E)  Pm(x,dy) 

A 

>  M>  (E)  Pm(x,A)  >  Xfxp  (E) 

so  that  {X^}  i-s  (S,  S,  X6,  <j>,  n+ra)  recurrent. 

«=)  For  such  a  chain,  the  uniformity  condition  of  Lemma  4.7  is 
trivially  satisfied,  and  hence  the  process  must  be  Doeblin  recurrent 
with  single  ergodic  set.  Also,  for  any  k _>  0, 

Pn+k(x,E)  -  /  Pk(y,E)  Pn(x ,dy) 

S 

>  X  /  Pk(y,E)  cp(dy) 

S 
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so  that  {X^}  is  (S,  S,  X,  cpP  ,  n+k)  recurrent.  This  clearly 
implies  that  txn)  is  aperiodic. 

Let  {Xq}  b®  an  aperiodic  Harris  chain.  We  have  seen  that  the 

uniformity  condition  manifests  itself  in  the  global  convergence  inherent 

in  (5.4).  We  now  examine  the  differing  nature  of  the  mixing  conditions. 

Put  Fn  =  8(X  ,  X  , . ,  ...).  Then,  if  (X  }  is  an  aperiodic 
n  irri  n 

Harris  chain  with  invariant  probability  it,  we  have  (Orey  [19]) 


li»  lP„(»n  B)  P„(B)I  -0 

k  -*■  ®o  B  €  F 

for  any  A  €  F  .  On  the  other  hand,  if  the  chain  also  satisfies  the 
uniformity  condition,  then  by  Lemmas  4.7  and  4.8,  {X^}  is  an  aperiodic 
Doeblin  chain  with  single  ergodic  set.  The  process  then  automatically 
possesses  an  invariant  probability  n ,  and  there  exists  constants  y 
and  p  (0  <  p  <  1)  such  that 


A  €  F 


m 


|Pn(A  D  B)  -  Pn(A) 


PTT  (») 


Ptt(A) 


<  TP 


(Billingsley  [3],  p.  168),  where  we  interpret  0/0  =  0. 

We  now  turn  to  the  question  of  strong  laws  for  Harris  chains.  Let 

[X  }  be  a  Harris  process  possessing  invariant  probability  n.  Then, 
n 

it  can  be  shown  that  if  tt  | f  l  <  +», 
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(5.9) 


Si.*  *<V  *’f 


a.s.  for  every  x  in  S  (see  Glynn  [12]).  This  has  the  important 
implication  for  simulation  that  one  can  estimate  rrf  by  forming  sample 
averages  along  a  single  sample  path  of  the  process. 

This  is  in  contrast  to  the  situation  that  occurs  when  one  forms 
m  independent  replicates  of  the  process,  and  simulates  to  time  n  along 
each  replicate.  This  gives  a  data  sample 


X1  X1 

Aj  9  **2  )  •  * 


x1  x2 

a2»  a2>  • 


A  «  A  t  •••  ,  A 

m  m  n 


Here,  the  estimate  is  formed  by  taking  sample  averages  along  each 
replicate,  and  averaging  the  replicates.  The  hope  is  that 


lim  lim  —  J  (i  l  f(Xk)] 
k=l'n  i-1 


converges  to  nf,  a.s.  for  every  x  in  S.  But  this  estimate 
tends  to 


(5.10) 


•i.  Ex(  l 
>  «  X  '  lc*l 


n  £(Xk) 


3 


and  this  is  not  in  general  equal  to  nf  for  a  Harris  chain. 


(5.11)  Example 

Let  P  be  the  transition  kernel  of  the  {W  }  chain  of  Example  2.2, 

n 

and  suppose  that  we  modify  it  to 

P(n,E)  -  6q(E)  +  j  6n+1(E)  ,  n  >  1 

P (x , E)  -  P(x,E)  ,  x  (  (1,  2,  ...}  . 

Clearly,  the  P  chain  is  still  Harris,  since  {0}  is  still  hit  i.o. 
Also,  the  modification  does  not  disturb  the  invariant  probability  n. 

We  now  introduce  a  non-negative  function  f  such  that  nf  <_  1  and 
modify  it  to 


f (n)  =  2n+1  ,  n  >  1 

i(x)  -  f(x)  ,  x  if  {1,  2,  ...} 

Then,  clearly  E,  f (X  )  >  f(n+l)  P  {X  =  n+1)  =  2  so  that  (5.10)  must 
In-  in 

A 

be  greater  than  nf. 

The  reason  that  the  convergence  problem  appeared  in  Example  5.11 

A 

lies  in  the  fact  that  f  was  not  bounded  (compare  with  (5.3)).  However, 
by  using  properties  of  f-regular  measures,  results  of  Nummelin  [17] 
imply  that 
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^  l  EKf'V”E 

for  it  a.e.  x  in  S.  Of  course,  the  simulator,  before  commencing 
the  simulation,  does  not  know  tt,  and  hence  the  above  result  is  only 
mildly  reassuring  to  those  that  wish  to  simulate  across  replicates . 


6.  Strongly  Recurrent  Harris  Chains 

In  this  section,  we  continue  our  study  of  (A,  B,  X,  <p,  1)  chains. 
In  particular,  we  shall  concentrate  on  simulation  questions  related 
to  such  processes. 

Let  us  start  by  noting  that  such  processes  Include  all  chains  which 
hit  a  given  point  xq  i.o.  for  we  can  take  A  «  {x^},  B  =  S,  X  =  1, 

<J>(')  =  P(Xq,  ’)•  Of  course,  other  processes  also  fit  into  this  category. 

(6.1)  Example 

We  say  that  a  GSMP  has  a  single  set  if  there  exists  a  state  s  with 
which  is  associated  only  one  clock.  In  this  case,  the  kernel  P  has  the 
form 


P((s,c) ,  A)  =  p(s';  s,i*)  n  F(a  ;  s',  i,  s,  i*) 

i€N  ,  x 
s 

so  that  the  right  hand  side  is  independent  of  the  clock  reading  c 
associated  with  s.  Given  that  the  GSMP  passes  through  (s)  i.o.. 
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the  chain  associated  with  P  is  (A,  B,  X,  cp,  1)  recurrent,  where 

A  =  {(s.c)},  B  -  S,  A  -  1,  <p(0  =  P((s,c),  •)• 

An  example  of  a  strongly  recurrent  chain  that  is  not  regenerative 

in  the  classical  sense  is  given  by  Example  2.9. 

As  demonstrated  in  Section  3,  these  chains  can  be  embedded  in  a 

regenerative  environment.  Recall  that  if  {X  }  is  an  (A,  B,  X,  <p,  1) 

n 

chain  on  S,  then  {X^}  can  be  viewed  as  a  component  of  the  regenerative 
process  =  (X^,  6^)  on  S  x  {0,1}.  By  simulating  X^,  one  can  then 
obtain  estimates  for  X^  in  a  straightforward  way.  We  now  outline 
the  procedure. 

Let  f  be  an  S-measurable  function  with  n | f |  <  -H»,  and  let 
r  ■  nf.  By  simulating  X^,  we  obtain  a  sequence  of  regeneration  times 
Nl’  N2’  *"*  We  then  form  the  sequences  (with  NQ  *  0) 

\  -  l  V(n+1) 

k*=0 

“n  '  J.  V(”+1) 

k=0 

and  note  that  the  regenerative  character  of  the  X^  process  guarantees 
that  the  sequence  {(Y^  ®^)»  k  >  1}  of  vectors  is  i.i.d.  The  classical 
regenerative  theory  then  asserts  that 

(6.2)  ?  -  Y/cf  +  r  P',  a.s. 

n  n  n  pi' 


vr1 


j=*N 


l  f(xj)  ; 


\  ”  Nk+l  ‘  Nk  ; 
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This  is  Che  well  known  ratio  estimator  of  regenerative  simulation 

theory.  Turning  to  the  question  of  confidence  intervals,  let  us  assume 
2  2  2 

that  a  »  E(Y^  -  ra^)  satisfies  0  <  cr  <  +».  Then,  the  i.i.d.  structure 
of  the  (Y^»  a^)  process  gives  us  the  central  limit  theorem  (CLT) 


(6.3) 


P' . 


1/2.-  , 

n  (Vr) 


a/a 


dt 


for  any  initial  distribution  p..  The  value  of  a  can  then  be  determined 
by  employing  standard  regenerative  estimates  (see  e.g.  Crane  and 
Lemoine  [7]).  Confidence  intervals  can  then  by  easily  calculated  from 
(6.3). 

The  limit  theorem  (6.3)  is  a  result  based  on  simulating  a  given 
number  of  cycles.  A  second  CLT  based  on  number  of  steps  simulated  can 
also  be  derived.  Let 


(6.4) 


“n  *  l  «V/(n+1) 

k=0 


Then,  it  can  be  shown  that  if  0  <  a  <  +«, 

1/2, 


(6.5) 


(  n  '  (U  -r)  )  .  x  2. 

■’.< - !L^<X  ->_i_  f  -t  n 

*  a/(  Ea)1/2-  j  V^T  - 


dt 


One  can  of  course  estimate  a  and  Ea^  using  the  point  estimates 
indicated  above. 
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Remark 


(6.6)  For  a  given  strongly  recurrent  chain,  there  may  exist  a 

multitude  of  (A,  B,  X,  <p,  1)  tuples.  Consider  two  such  tuples 

(A^,  B^,  X1#  <p^»  1),  i  ■  1,  2.  The  values  of  a  and  Ea^  will  in 

2  2 

general  depend  on  i.  However,  (6.5)  shows  that  if  0  <  ct^,  <  +°», 
then 


(6.7) 


Ei“i 


E2®2 


In  other  words,  the  ratios  of  (6.7)  are  essentially  independent  of  the 
choice  of  (A,  B,  X,  <p,  1)  tuple. 


Our  next  objective  is  to  investigate  the  choice  of  (A,  B,  X,  <p,  1) 
tuple  from  a  simulation  viewpoint.  We  suppose,  for  purposes  of  the 
present  discussion,  that  A  has  been  fixed.  One  now  observes  that  the 
set  B  is  irrelevant  in  this  context,  as  a  process  that  is  (A,  B,  X,  cp,  1) 
recurrent  is  clearly  (A,  S,  X,  ip,  1)  recurrent,  where  <p  is  the 
extension  of  <p  given  by 


(6.8) 


tp  (E)  «  <p(E  n  B) 


A. 

We  will  not  distinguish  between  <p  and  tp  in  what  follows. 

Commonly,  a  simulator,  in  applying  the  regenerative  method,  picks 
a  "regeneration  point"  with  the  aim  of  maximizing  the  expected  number 
of  regenerations  over  a  fixed  number  of  transitions.  Now,  in  our  case, 
the  expected  number  of  regenerations  over  [0,n]  is 
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*ii  i(xk-i'A’6k*ii>,'n 


ix  %•  (xk-i  *  a-  \  ■  i,/n 


n 

X  l  P \{X  €  A}/n  *  Xn  (A) 

k-1  * 


so  that  finding  a  (X,<p)  pair  which  maximizes  X  and  for  which  the 
chain  is  (A,  S,  X,  cp,  1)  recurrent  appears  to  be  a  desirable  objective. 


(6.9)  Lemma 

Given  that  a  chain  is  (A,  S,  X,  cp,  1) 

A  A  * 

a  (X.cp)  pair  which  maximizes  X  over  all 


recurrent,  there  exists 
(A,  S,  X,  <p,  1)  tuples. 


Proof 

Let  X  be  the  X  sup  taken  over  all  (A,  S,  X,  cp,  1)  tuples. 

A 

Our  goal  is  to  show  that  there  exists  a  measure  ip  such  that  the  chain 

A  A 

is  (A,  S,  X,  cp,  1)  recurrent. 

Take  a  sequence  of  Harris  tuples  (A,  S,  X  ,  cp  ,  1)  such  that 

n  n 

X  *  X.  Put 
n 

r-dcp  (•)  i 

gn(x,y)  -  \J_dPuryJ (y) 

h  (x,y)  =  max  g.  (x,y)  • 

n  1  1  k  1  n  K 

Now,  h  is  an  increasing  sequence  of  functions  so  a  limit  function  h 
n 

exists.  Define  the  measure  \^  (•)  by  che  formula 
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+V(E)  -  /  h(x,y)  P(x,  dy)  . 
X  E 


We  shall  show  that 

A  A 

i)  (*)  Is  independent  of  x.  Denote  the  common  value  by  Xp(*)- 

A  A 

ii)  (A,  B,  X,  <p,  1)  is  a  legitimate  Harris  tuple. 

For  i) ,  observe  that  it  is  sufficient  to  show  that  for  each  integer  n, 


/  h  (x,y)  P(x,  dy)  =  /  h  (z,y)  P(z,  dy) 
E  n  E  n 


for  all  x,  z  €  S.  We  prove  this  fact  by  induction.  For  n  =  1,  we  have 


/  hj^x.y)  P(x,  dy)  =  Xj^  q>1(E) 
E 


which  is  independent  of  x.  Assume  the  validity  of  the  induction  for 
n  =  k,  and  put 


t  (E)  =  /  h  (x,y)  P(x,  dy) 
E  K 


Then,  by  letting  v  be  the  signed  measure  (•)  -  X^+^  <p  ( •)»  we 
can  apply  the  Hahn  Decomposition  Theorem  (Royden  [21],  p.  235)  to 
assert  existence  of  a  set  F  such  that  F  is  a  positive  set  (the 
v-raeasure  of  every  subset  is  non-negative)  and  F  is  a  negative  set. 
Since  both  f  (•)  and  Xj^  ^k+1^’^  are  abs°lutely  continuous  w.r.t. 
each  of  the  measures  P(x,-)»  this  implies  that 


\(x,y)  -  Sk+l^x,y^  2  0  >  P(x.-)  a.e.  on  F 


hk(x>y)  ~  8k+l^X,y^  —  0  P(x»')  a.e.  on  FC 

Now,  hk+^(x,y)  ”  max^k^x,y^’  ®k+l^x,y^  so  c^at  ^or  eac^  x»  we  have 

/  hk+1(x»y)  p(x»  dy) 

G 

=  /  h  (x,y)  P(x,  dy)  +  /  g.  , (x,y)  P(x,  dy) 

EOF  K  EflF  K 

=  tk(E  n  f)  +  xR+1  <Pk+1(E  n  FC) 

which  is  independent  of  x,  proving  i) .  We  now  turn  to  ii) .  We  first 
note  that  by  Monotone  Convergence, 

Xtp(S)  =  /  h(x,y)  P(x,  dy) 

S 

=  lim  /  h  (x,y)  P(x,  dy) 
h  -*■  «  S 


2  lim  max  /  g  (x,y)  P(x,  dy) 
n  +  ”l<k<n  S 

A 

=  lim  X  =  X 
_  .  _  n 


so  that  <p(S)  2  !•  We  now  observe  that  by  Remark  3.3,  0  h(x,y)  <_  1 
P(x,-)  a.e.  so 


p(y)  =  inf  p(x,y)  =  inf  p(x,y) 
x  €  A  x  €  C 

so  that  p('),  being  the  infinum  of  countably  many  functions,  is 
measurable. 

To  show  that  \|c  is  a  feasible  solution,  it  is  necessary  to  prove 
that  'lf(-)  _<  P(x,*)  for  each  x  in  A.  But  this  follows  as  a 
consequence  of  the  inequality 


/  p(y)  P-(dy)  «  /  inf  p(x,y)  n(dy) 

E  E  x  €  A 

<  inf  /  p(x,y)  u(dy)  . 
x  €  A  E 

As  for  the  optimality,  let  T  be  any  other  finite  measure  satisfying 
't(-)  _<  P(x,-)-  Then,  it  is  easily  checked  that 


x.(E)  -  t(E  n  I)  <  l  a  5  (E)  , 
t  z  ^  I  z,x  z 

(6.11) 

t„(E)  =  T(E  (1  Ic)  <  /  p(x,y)  p.(dy)  , 

E 

For  t^,  this  clearly  implies  that 


V  x  €  A 


V  x  €  A  . 


(6.12)  VE)  1  I  S  6(E) 

As  for  Tj,  we  have  that  <<  p.  so,  letting  Tj(y)  be  the  derivative 
of  i 2  w. r.t.  n,  we  have 
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^2(y)  <  p(x,y)  n  a.  e. 

for  each  x  €  A.  But  this  means  that 

w  •  {y1^^  —  p<x»y>  v  x  e  c> 

"  {y:T,(y)  £  inf  p(x,y)} 
x  €  C 

has  p,(WC)  =  0.  Hence,  T2(y)  .1  PW  pa.e.  This,  together  with  (6.11) 
and  (6.12),  implies  that  t(E)  £\jf(E),  and  so  ^  is  optimal. 

A  A  A  A  A 

To  recover  X,  <j>,  put  X  =  t(S)>  <?(•)  =  t(')A. 

Remarks 

(6.13)  The  above  discussion  kept  A  fixed.  If  a  simulator  has 
several  candidates  A^  for  A,  then  (at  least  theoretically)  one 
could  run  trial  simulations  to  estimate  n(A.),  and  determine  the 

l 

optimal  X^  for  each  of  the  A^.  One  could  then  choose  the  A^ 

a 

maximizing  X^  n(A^)  as  the  "best"  A. 

(6.14)  One  could  further  increase  the  frequency  of  regeneration  by 
employing  the  followng  method.  Let  q^(y)  be  c^e  derivative  of  ip 
w.r.t.  P  (x ,  • )  ,  and  define 


f  -  »V->" 


X 
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so  that  P(x,E)  _>  X  <p(E).  Observe  that  X  >_  X  for  every  x  in  A, 
and  that  the  decomposition 

P(x,E)  =  X  <J(E)  +  (1-X  )  Q(x,E) 

x  X 

is  still  possible.  One  could  now  follow  precisely  the  method  of  (3.9) 
to  (3.10)  to  embed  {X^}  in  a  regenerative  M.C.  X' .  The  asymptotic 
frequency  of  regeneration  for  this  chain  would  be 

/  X(x)  tt(dx) 

S 

A 

which  is  clearly  greater  than  Xn(A). 

Example  2.8  continued 

Consider  the  M/M/1  version  of  the  {W^}  chain.  Here,  the  density 
f  is  given  by 


a  ay 
(a+1)  * 


f(y)  = 


. a.  e-y 

(a+l)  e  ’ 


y  <  0 


y  >  0 


where  0  <  a  <  1.  Consider  a  recurrence  interval  of  the  form  A  =  [Q,b]  , 
b  >  0.  All  of  the  criterion  of  our  optimality  discussion  are  satisfied 

AA 

so,  putting  \jr  *  hp. 
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t(E)  =  6q(E)  F(-b)  +  /  p(y)  dy 


where 


p(y)  *  inf  {f(y-x)} 
x  €  [0,b] 


inf  {f (y)} 

u  €  [y-b,y] 


•  min{f (y-b) ,  f(y)} 


g 

(a+1) 


g(y-b) 
e  » 


0  <  y  < 

—  3  —  a+1 


a 

(a+1) 


y  > 


ab 

a+1  * 


Of  course,  X  =  F(-b)  +  /  p(y)  dy,  and  <(>(•)  =  \|/(-)/X. 

0 

The  measures  Q(x,*)  can  also  be  calculated.  Defining 
through 


H,  h 


(1-X)  Q(x,E)  =  H(x)  6  (E)  +  /  h(x,y)  dy 

E 

we  see  that  H(x)  =  F(-x)  -  F(-b),  h(x,y)  =  f (y-x)  -  p(y).  For 
0  _<  x  £  Xb/(X+1),  h(x,y)  takes  the  form 
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-ax  -ab, 
e  -  e  ]  , 


ae 


ay 


h(x,y) 


(a+1)  =  J  » 

—  re"(y_x)  -  ea(y~b>l 

a+1  le  e  j  , 


0  <  y  <  x 
.  ,  Xb 


-y 


ae 

a+1 


[e  -  1]  , 


.  Xb 


The  function  h(x,y)  takes  on  an  equally  complicated  form  over  each 
of  the  intervals  [Xb/(X+1),  b] ,  [b,  +») . 

The  above  example  indicates  that  the  computation  involved  in 
calculating  Q(x,*)  may,  in  general,  be  formidable.  However,  from  a 
simulation  stand  point,  explicit  knowledge  of  Q(x,’)  is  unnecessary 
One  can  always  obtain  deviates  distributed  according  to  Q(x,*)  by 
utilizing  the  following  acceptance-rejection  method. 

We  assume  that  the  assumptions  on  S,  A,  and  P  required  to 
obtain  (6.10)  are  in  force.  The  first  step  is  to  obtain  a  deviate 
Z  distributed  according  to  P(x,*)>  One  then  puts 


(6.15) 


P(Z) 


a  /a 
z  z  ,x 


p(Z)/p(x,Z)  , 


if  Z  =  z  €  I 

x 


otherwise 


and  generates  a  Bernoulli  random  variable  I  such  that  P{I  =  l|Z} 
=  p(Z)  =  1  -  q(Z).  We  claim  that 


P{Z  €  E|l  «  1}  =  cp (E) 


P{Z  €  Eli  -  0}  =  Q(x,E)  . 

To  see  this,  observe  that 

P{Z  (  E,  I  ■  1} 

-  p{z  €  e  n  i  ,  i  -  i}  +  p{z  e  e  n  ic,  i  -  i} 

■  I  a  P{I  =  1 1 Z  =  z}  +  /  P{I  =  1 1 Z  =  z}  p(x,z)  p(dz) 

z€EflI  Z*  E 

x 

”  l  a  +  /  p(z)  p.(dz)  =  t(E) 

Z€E Dl  z  E 

x 

so  that  P{I  =  1}  -  P{Z  €  S,  I  =»  1}  =  t(S)  =  x.  Hence,  P{Z  €  Eli  -  1}  =  (p(E). 
For  Q(x, • ) ,  note  that  P{Z  €  E,  I  =  0}  =  P{Z  €  E}  -  P{Z  €  E,  1=1} 

A  A 

=  P(x,E)  -  ^(E)  =»  Q(x,E)  (1-X),  and  that  P{I  =  0}  =  1-X. 

Our  final  comnents  in  this  section  focus  on  implementation  of  the 
above  method  in  an  actual  simulation.  The  method  requires  a  simulator 
to  determine  a  Harris  tuple  (A,  B,  X,  cp,  1).  The  simulator's  intuition 
for  the  model  to  be  simulated  would  be  a  valuable  tool  in  obtaining 
appropriate  choices  for  the  set  A.  One  would  have  to  consider  whether 
the  set  A  is  hit  infinitely  often,  as  well  as  whether  the  kernel 
could  be  expected  to  be  reasonably  smooth  over  A. 
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The  determination  of  X  and  <p  is  a  more  exacting  matter. 

While  the  mathematical  conditions  involved  in  applying  (6.10)  to 
finding  X  and  <p  are  fairly  mild  (from  a  practical  standpoint), 
such  an  application  does  require  an  explicit  knowledge  of  the  kernel 
P,  over  the  set  A.  However,  most  simulations  of  complex  stochastic 
systems  are  generated  through  an  event  scheduling  approach  that  does  not 
require  an  explicit  form  for  the  transition  probabilities  (see  e.g.. 
Crane  and  Lemoine  {7],  p.  72-83).  Therefore,  the  simulator  will 
generally  be  faced  with  the  task  of  determining  the  structure  of  the 
kernel  P,  a  task  which  could  potentially  be  quite  time  consuming. 
However,  it  is  to  be  emphasized  that  an  explicit  form  for  P  is 
required  only  over  the  set  A. 

Once  X  and  tp  have  been  found,  simulating  the  chain  would 

require  only  a  moderate  additional  amount  of  work.  To  obtain  X^,  one 
would  generate  a  sample  path  of  the  process  Xn  in  the  normal  way. 

At  each  transition  of  X^,  one  would  also  generate  a  Bernoulli  random 
variable  6n  with  parameter  p(X^).  The  value  of  p(x)  is  given  by 

Q 

(6.15)  over  the  set  A,  while  its  value  over  A  is  X  identically. 

The  pair  (X  ,  6  )  would  then  constitute  the  process  X'. 

n  n  n 


7.  Weakly  Recurrent  Harris  Chains 

We  shall  say  that  a  Harris  chain  is  weakly  recurrent  if  it  is  not 

strongly  recurrent.  In  this  section,  we  shall  investigate  a  means 

for  embedding  a  weakly  recurrent  chain  in  a  process  that  can  be 

statistically  analyzed  in  a  more  manageable  way. 

In  Section  3,  we  gave  an  example  (Example  3.17)  of  a  process  which 

is  weakly  recurrent.  We  now  give  a  less  artificial  example.  Consider 

the  chain  associated  with  a  GSMP  that  contains  no  single  sets,  and 

suppose  that  the  process  is  (B^,  B^,  a,  \|r,  1)  recurrent.  For  each 

(s,c)  in  the  state  space  S  of  the  chain,  the  kernel  P((s,c),  •) 

contains  an  indicator  function.  Then,  since  f  «  P((s,c),  •)  for 

some  (s,c),  \|r  must  be  supported  on  that  part  of  S  for  which  the 

* 

clock  i  €  0  ,  is  set  at  the  value  c  (see  (2.11)).  Clearly,  if  \|f 

S  X 

is  supported  on  such  a  set,  the  chain  must  pass  through  the  set  infinitely 
often.  But,  in  general,  a  clock  will  never  be  set  stochastically  at 
the  same  value  twice  and  hence  the  chain  cannot  visit  such  a  set 
infinitely  often.  Hence,  the  chain  associated  with  a  GSMP  without 
single  set  will  generally  be  only  weakly  recurrent.  This  motivates  our 
present  discussion. 

As  discussed  in  Section  3,  the  randomization  P  of  a  weakly 
recurrent  kernel  P  with  Invariant  probability  rr  is  strongly 
recurrent.  If  one  were  interested  in  an  estimate  of  irf  for  the 

A 

kernel  P,  one  could  then  simulate  the  chain  associated  with  P  using 
the  techniques  of  Section  6  for  strongly  recurrent  chains  to  obtain  a 
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confidence  interval.  The  most  natural  way  to  simulate  the  randomized 

chain  would  be  to  employ  its  interpretation  as  a  random  sampling  of 

the  original  chain  (see  Remark  3.19).  In  other  words,  one  would 

simulate  the  original  chain  {Xn},  as  well  as  an  independent  i.i.d. 

sequence  {Z^}  of  geometric  random  variables  with  parameter  p.  Then, 

by  putting  =  n  +  Z^  +  +  •••  +  Z^,  the  process  {X^  }  would 

n 

have  exactly  the  distribution  of  the  randomized  process.  The  difficulty 

is  that  one  only  retains  a  proportion  p  of  the  {X  }  sample  path  in 

n 

forming  confidence  intervals  based  on  the  {X^  }  chain.  Since  this 

n 

is  clearly  undesirable,  we  examine  a  second  method  of  simulating  weakly 
recurrent  chains . 

We  shall  show  that  under  mild  assumptions  on  (S,S),  a  weakly 

recurrent  chain  {X  }  can  be  embedded  in  a  process  that  is  loosely 
n 

regenerative  in  the  sense  of  Smith  [22].  We  shall  be  more  precise 
about  this  later. 

Let  the  weakly  recurrent  chain  {X  }  be  (A,  A,  X,  tp,  n)  recurrent. 

n 

We  start  by  considering  the  process  Z^  =  •••»  xn(m+i)^  which 

lives  on  the  state  space  Sn,  equipped  with  product  a-field  Sn,  and 
transition  kernel  K  given  by 

(7.1)  K((Xl,  ....  xn),  E1  x  •••  x  En) 

=  /  P(x  ,  dy.)  /  P (y,  dy  )  •••  /  P(y  ,,  dy  )  . 

Ex  E2  En 
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Then,  for  an  initial  probability  a  on  (Sn,  Sn) ,  we  can  construct  a 

probability  tuple  (S2n,  Fn,  K  )  for  which  the  random  variables  Z 

u  m 

are  obtained  by  projection  (see  Section  2). 

The  original  chain  {X^} »  which  has  initial  probability  P-  (say) , 
is  recovered  by  setting 


(7.2)  p,(E  x  ...  x  E  )  =>  /  ji(dx)  /  P(x,dy  )  •••  /  P(y  . ,  dy  ) 

1  n  S  E.  1  E  n_1  n 

1  n 

The  process  {Z^;  m  >  0}  governed  by  then  corresponds  precisely 

to  {X^}  under  P  .  Now,  it  can  be  shown  (Athreya  and  Ney  [1])  that  if 
{X^}  is  initiated  with  probability  p.  concentrated  on  A,  then  the 
process  X^  visits  A  infinitely  often.  This  clearly  implies  that 
the  process  Z^  visits  Sm  ^  x  A  infinitely  often  under  K— . 

We  assume  for  the  remainder  of  this  section  that  S  is  a  complete, 
separable  metric  space,  with  Borel  o-field  S.  Now,  we  have  that 


P  (x,E)  >  Vp(E) 


for  every  x  in  A  and  E  in  A.  Again,  we  can  perform  the  decomposition 


P  (x,E)  =  \p(E)  +  (1-X)  Q(x,E) 


where  Q  is  a  transition  kernel  on  (S,  S) .  We  want  to  perform  a 


similar  decompositon  on  the  kernel  K.  Now,  since  S  is  a  complete, 
separable  metric  space,  (Sn,  Sn)  is  a  Borel  space  (see  Breiman  [5], 
p.  79,  402)  and  so  a  regular  conditional  probability 


(7.3) 


VX1  €  El* 


X  .  €  E  . |X  =  y) 
n-1  n-1  n 


exists  for  every  y  in  S.  That  means  that  for  each  y  €  S,  (7.3) 
is  a  probability  on  (Sn  \  Sn  ^),  whereas  for  each  tuple  (E^, . • • ,En_^) , 

(7.3)  is  a  S-measurable  version  of  the  conditional  probability.  Define 
the  family  of  measures  ip(x,*)»  Q(x,')  by  the  formulas 


(7.4)  tp (x ,  Ex  x  E2  x  •••  x  En) 


-  '  tJh  6  E1 . X„-l  E  En-X,Xn  ’  y)  ’,(<ly) 

E 

n 


Q(x,  Ej  -  E2  x  ...  x  En) 


-  >  px(lti E  Ei . Vi {  Vilx»  ■ y)  Q(Xl  dy) 

E 

n 

for  each  x  in  A.  Observe  that  for  each  (x  ,  ...,  xr)  f  s"  1  *  A, 
we  have 

(7.5)  K((xx,  ....  xn),  Ex  x  •••  x  En) 


=  \ip(x  ,  El  x  •••  x  Efl)  +  (l-\)  Q(xn,  Ex  x  •••  x  Er) 


We  can  now  describe  the  embedding  for  Z^.  Let  S'  =  S  x  {0,1}  and 
let  S'  be  the  associated  product  CT-field.  Define  a  kernel  K' 
relative  to  (S',  S')  by 
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^(((x.,  ....  x  ),  6),  E.  x  ...  x  E  x  {0}) 

I  n  1  n 

(1-X)  K((x. ,  ....  x  ) ,  E  x  • ••  x  E  ) ,  (x  , . . . ,x  )  i  Sn  1  x  A 
1  n  1  n  J.  n 

a 

(1-X)  Q(x  ,  E.  x  E,  x  • • •  x  E  )  ,  (x,,...,x  )  € s"  1  x  A 

n  i  z  n  i  n 


K'(((x  ,  . . . ,  x  ) ,  6) ,  E1  x  *..  x  E  x  {1}) 

Ini  n 

XK((x1,  . . . ,  x  ) ,  E  X  • • *  x  E  )  ,  (x  , ...,x  )  £  Sn  1  x  A 

1  n  1  n  In 

Xip(x  ,  E.  x  •  x  E  )  ,  (x  , . . .  ,x  )  6  S  x  A 

n  1  n  in 

We  extend  measures  a  on  (Sn,  Sn)  to  measures  cr'  on  (S',  S’)  as 

in  (3.4).  Then,  by  letting  Z'  =  (Z  ,  6  )  be  the  canonical  M.C. 

'  ra  mm 

corresponding  to  S'  and  K' ,  we  see  that 

i)  for  any  a  on  (Sn,  Sn) ,  the  marginal  distribution  (w.r.t.  K^,) 

of  the  coordinate  process  Z  of  the  M.C.  Z'  is  the  same  as 

m  m 

that  of  the  original  chain  Z  (w.r.t.  K  ). 

m  g 

ii)  the  random  variables  {5^}  constitute  an  i.i.d.  Bernoulli  sequence 
with  parameter  X. 

Define  N,  *  inf{m  >1:Z  ,  €  Sn  ^  x  A,  5  =1}  and  put 

l  —  m—  J-  m 

N.  ■  inf{m  >  N.  . :Z  €  Sn  ^  x  A,  5  =  1}.  We  first  note  that  there 

k  k-1  m-1  m 

are  an  infinite  number  of  N^»  provided  a  =  n,  where  p.  is  concentrated 
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on  A.  Since  our  discussion  is  motivated  by  ergodic  properties  of 
{Xn},  it  is  no  restriction  to  only  consider  such  p,  for  by  (5.9)  the 
initial  position  washes  out  in  the  limit. 

We  now  consider  the  sequence  of  random  vectors 


+1’  *  ‘ 4  ’  ^  ’ 

K  V1  \+l 


k  >  1  . 


Now,  the  sequence  {Z^;  k  ^  1}  is  identically  distributed.  By 

the  Strong  Markov  property,  this  will  follow  if  the  sequence  consisting 

of  the  first  component  of  is  identically  distributed.  The 

k 

distribution  of  this  component  is  given  by 


(7.6) 


Kl'{Xn(N  +1)+1  *  E}=  ;  P(y’E) 

p  k  S 


proving  our  claim.  However,  the  sequence  of  vectors  Z^  is  not 

independent  as  the  distribution  of  Z^  clearly  depends  on  that  of 

k 

Z  .  It  is  true,  though,  that  the  sequence  {Z  ;  k  ^  1}  is  1-dependent. 

k  A  k 

In  other  words,  {Z  ,  ...,  Z  }  is  independent  of  {Z  7 ,  . . . ,  Z,  } 

X  x  tc  •  x  ictto 

for  m  _>  1.  This  is  basically  a  consequence  of  the  independence  of 

Z^  and  Z  ,  which  we  now  endeavor  to  prove.  Evidently 
k  Nk 


Kl’<V-u  £  ElVi> 

p  k  k 


p  p  k'  k  k 
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where  F*  is  the  o-field  generated  by  the  stopping  time  N  along 
k  k 

the  Z’  sample  path.  By  the  Strong  Markov  property,  the  above  is  just 


E\{K^t  {Z1  €  E}|  ZN  _x> 


But  observe  that  the  interior  probability  depends  on  Z'  only  through 

\ 

X and  this  later  random  variable  always  has  distribution  <p(’)» 
completing  the  argument. 


Remark 

(7.7)  The  existence  of  a  sequence  of  stopping  times  for  which 

ZN  and  Z^  +1  are  independent  guarantees  that  {Z^}  is  loosely 
k  k 

regenerative  in  the  sense  of  Smith  [22].  To  incorporate  such  a  process 

into  his  framework,  consider  the  Z^  process  at  each  of  the  stopping 

times  *  N^+l.  Then,  the  history  of  the  process  up  to  time  N^~2 

has  no  value  in  predicting  the  future  evolution  of  Z  ,  and  so  this 

m 

process  is  loosely  regenerative. 

We  will  now  obtain  a  CLT  for  the  Z^  process.  Let  f  be 

S-measurable  with  tr|f  |  <  +»,  where  tt  is  a  probability  assumed 

invariant  for  (X  }.  Extend  f  to  a  function  f  on  Sn  by  the 
n 

formula 


f(Xl,  ....  xn)  -  n  ^  f(xi)  . 
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Then,  putting  r  =  rrf,  we  can  define  the  sequences 


Nk+1 

\  -  l  f(z.) 

j=Nk+l  J 


\  =  \+l  "  \ 


V,  =  Y,  - 
k  k 


ra. 


which  we  note  are  all  1-dependent.  Also,  since  the  sequence  {Zk}  is 
identically  distributed,  all  three  of  the  above  sequences  are  strictly 
stationary.  Furthermore,  the  sequence 


a 


k 


i 


is  merely  the  sequence  f (X .)/£}  stopped  at  the  random  times 

1 

Hence  by  the  nature  of  a.s.  convergence,  it  must  be  that 


nN 


m+1 


tn  m 

(7.8)  rn=  l  Y  /l  ak  -*■  r  K\  a.s. 

11  [L 

-2  2 

It  should  be,  then,  that  E' ,  Vk  =  0.  Thus,  given  that  a  =  E' ,  Vk 

-2  ^  ^ 
satisfies  0  <  a  <  +°°,  we  can  apply  Theorem  20.1  of  Billingsley  [3]  for 

cp-mixing  processes  to  obtain  the  C.L.T. 
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where  a 

(7.9) 


El»  Vk  +  2E1*  vt  Vir+i-  Consequently 

p,  M- 


^ 

(a/a) 


also  satisfies  a  C.L.T.,  where  a  is  the  sample  average  of  {a^}.  To 

apply  (7.9)  to  confidence  interval  estimation,  it  would  be  necessary 
2  2 

to  estimate  o  .  Expanding  a  as 


(7.10)  V,  {Y2  -  2ro^  +  r2a£  ♦  Y  Y  -  Y  -  r«  Y  +  r2Vk+1> 

we  see  that  each  term  can  be  estimated  in  a  fairly  straightforward 
way.  Results  (7.8)  through  (7.10)  then  form  the  basis  for  a  confidence 
interval  estimation  technique  based  on  the  process. 

The  above  argument,  while  somewhat  loose,  points  to  a  simulation 
technique  for  estimation  of  "steady-state"  values  nf.  We  now  briefly 
discuss  implementation  of  the  method. 

A  first  step  would  be  to  determine  the  (\,cp)  pair  corresponding 
to  the  tuple  (A,  A,  X,  tp,  n).  Given  that  an  explicit  representation 
of  Pn  was  available  over  the  set  A,  one  could  find  (X,cp)  through 

formula  (6.10),  provided  that  the  assumptions  on  A  and  Pn  required 

\ 

for  validity  of  (6.10)  were  in  force.  While  we  have  already  mentioned 
in  Section  6  the  difficulties  in  finding  P  explicitly,  the  problems 
to  be  surmounted  in  doing  the  same  for  Pn  would  surely  be  even  more 
formidable.  This  is  because  the  simulator  would  have  to  follow  the 
process  through  n  transitions  in  order  to  find  Pn,  and  this  could 
conceivably  be  a  very  complex  task. 
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We  now  turn  to  generation  of  sample  paths  for  the  process  Z^,  given 

knowledge  of  a  (\,<p)  pair.  The  simulation  would  begin  at  a  point 

picked  from  the  set  A.  The  simulator  would  then  generate  a  sample 

path  for  the  original  process  {X  }  in  the  usual  way.  However,  were 

m 

X  ^  to  b®  in  A,  one  would  record  its  position  x  in  that  set.  The 
simulation  would  then  continue  to  run  in  the  normal  fashion  until 
Xn(k+i)  was  generated.  The  acceptance-rejection  method  of  (6.15) 
would  then  be  employed  in  order  to  determine  whether  Xr ^  came 
from  <p  or  Q(x,-)-  Given  that  X^ (k+1 )  was  distributed  according  to 
ip,  would  be  set  to  k.  The  above  chain  of  events  would  then  repeat 
itself  to  generate  the  complete  sequence  ,  ...  . 

The  final  step  for  the  simulator  would  then  be  to  use  (7.8)  through 
(7.10)  to  form  a  confidence  interval  for  the  quantity  rcf  of  interest. 


Our  objective  here  is  to  show  that 

P>A  <  +">  -  1 

A 

for  every  x  in  S,  where  P  is  the  randomized  version  of  a  Harris 
kernel  P  (with  Invariant  probability  it)  and  A  €  S  is  a  set  such  that 

Px{TA  <  *~}  “  1 

for  every  x  in  S.  We  start  with  a  couple  of  definitions.  We  say 
that  A  €  F  is  an  invariant  event  if  0  '''(A)  =*  A,  and  denote  the 
a-field  of  invariant  events  by  I.  If  Z  is  I-measurable ,  one  calls 
Z  an  invariant  random  variable.  Finally,  a  S-measurable  function  h 
is  said  to  be  P-harmonic  if 

h(x)  -  /  h(y)  P(x,  dy) 

S 

for  every  x  in  S. 

(A.l)  Lemma 

* 

Let  Z  be  a  bounded  invariant  random  variable.  Then,  h(x)  -  E^Z 


is  P-harmonic. 


Proof. 


Since  Z  is  invariant,  Zo0^  =  Z  for  all  j,  and  so 


E  Z  =  E  (ZoQ.) 

X  X  J 


■  E*fEXjZ1 


A  A  • 

/  E  Z-PJ(x,  dy) 


by  the  Markov  property.  Now,  it  is  easy  to  calculate  that 


(A.  2)  E  Z  =  l  (j+”"1)  pJ(l-p)n  /  E  Z'P^  (x,dy) 

X  n=0  S  y 


Put 


a  =  /  E  Z-Pi(x,  dy) 

S  y 

(A.  3) 

f(X)  -  a1  +  a2\  +  a3\2  +  •••  . 


Clearly,  since  | a^ !  _<  ||Z((  for  all  i,  the  radius  of  convergence  of  f 
is  at  least  one,  and  hence  f  is  analytic  on  any  interval  [— t , t ]  for 
t  <  1.  Hence,  for  any  X  €  (-1,  1),  we  have  for  j  >_  1 


f  <J)(X) 


-  I 

n*0 


Is+ilL 


n+j+l 


so  that  for  p  >  0  (putting  q  ■  1-p) 


,  ( j),  .  ,1 !  r  /j+nN  j+1  n 

£  (,)  '  jo  <  »  ’  P  q  W 


P 

1L 

J 


4  •  * 

7±t  E  Z 
j+1  x 


f  (q) 


(by  (A. 1)) 

(by  (A. 2) ,  (A. 3)) 


But,  since  f  is  analytic  at  q. 


CO  J 

f  (q+h)  =  l  f(j)(q)  JT  =  f  (q)  (^) 


j=0 


P~h 


which  is  valid  at  least  for  |h|  <  p.  Thus 


(A.  4) 


f(\)  =  f  (q) 


is  valid  for  q-p  <  X  <  1.  But  f  is  analytic  on  any  subinterval  of 
(-1,  1),  so  the  representation  (A. 4)  is  valid  at  X  *  0.  So, 


f(0)  -  pf(q> 


i.e. , 

/  E  Z  *  P  (x ,  dy)  -  E  Z  .  || 

S  y  X 


Now,  it  can  be  shown  that  every  bounded  P-harmonic  function  h 

A 

has  the  form  h(x)  ■  E^Z  for  some  I-measurable  Z  (Revuz  [20],  p. 
Hence,  Lemma  A.l  can  be  reformulated  as: 


82). 
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Every  bounded  P-harraonic  function  h  is  P-harmonic  . 


(A. 5)  Lemma 

The  bounded  P-harmonic  functions  are  constant. 

Proof. 

Let  h  be  a  bounded  P-harmonic  function.  Then,  by  (5.3) 

1  n  t 

h(x)  =  ‘TLi’  I  (p  fHx)  •*'  -T  h(y)  tt  (dy) 
n  1  k=0  S 

so  that  h  is  identically  constant.  || 

a 

Hence,  the  bounded  P-harmonic  functions  are  constant.  This  is 
known  to  imply  that  every  set  is  either  recurrent  or  transient  for 

A 

P  (Revuz  [20],  p.  83).  A  set  A  is  said  to  be  recurrent  (transient) 
for  P  if  ?x(xn  €  A  i.o.}  =  1  (=  0)  for  every  x  €  S. 

Now,  existence  of  an  invariant  probability  it  for  P  has  been 

A 

assumed.  This  measure  it  is  then  trivially  invariant  for  P,  and  so  0 

A 

is  a  measure  preserving  transformation  on  the  probability  space  02,  F»  P^). 
Then,  a  well-known  ergodic  result  (see  e.g.  Lamperti  [15],  p.  91)  states 
that 


P  {X 

TT 


n 


€  A  i.o.}  >  tt  (A) 


>  0 


A 

implying  that  A  must  be  a  recurrent  set  under  P. 
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Remark 


(A. 6)  Note  that  existence  of  an  invariant  n  for  P  is  sufficient  for 
the  above  argument  to  be  valid,  for  then 

rr(A)  =  /  P(x,A)  ff(dx) 

S 

We  now  observe  that  since  {X  }  hits  A  P  a.s.,  it  must  be  that 

n  x 

P(x,A)  is  positive  for  every  x  in  S,  and  consequently  n (A)  is 
positive.  This  is  the  only  modification  required. 
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